🕷️️ Job Radar • SCRAPING

upwork.com 🟡 2026-05-07

🔹 [Target] Web Data Crawling [Method] Design & Implementation [Stack] Python, Playwright, AWS [Security] Compliance with TOS and DLP [Format] JSON
👤 Client: TUR Member since 2020-11-03
💰 Price: ****
🚩 Problem: Design a resilient web data acquisition solution for collecting publicly accessible information from designated websites.
📦 Existing: Not specified

Specifications:

[Target] Design and develop a robust, scalable crawling framework
[Method] Extract structured data from dynamic, JavaScript-rendered websites
[Stack] Use Python (or equivalent) with asynchronous programming for efficient data collection
[Security] Ensure compliance with relevant terms of service and data usage policies; implement mechanisms to handle session persistence, rate limits, and adaptive traffic management
[Format] Deliver collected data in JSON format

Workflow:

1. Analyze target websites' architecture and identify key data points for extraction.
2. Develop a Python-based crawling framework using Playwright or equivalent tools to handle dynamic content and structural variability.
3. Implement intelligent retry, throttling, and session management mechanisms to ensure system stability and performance optimization.
4. Collaborate with internal teams to define data schemas and delivery requirements.
5. Deploy the solution in a cloud environment (AWS, GCP, or Azure) for scalability and high availability.